Online Learning Meets Optimization in the Dual

We describe a novel framework for the design and analysis of online learning algorithms. Our framework is based on a new perspective on relative mistake bounds by viewing the number of mistakes of an online learning algorithm as a lower bound for an optimization problem. This interpretation of a mistake bound draws a connection between online learning and optimization through the theory of duality. In particular, we show that any procedure which incrementally increases a dual objective function can be utilized for the design of an online learning algorithm. Our framework produces the best known mistake bounds for previously defined algorithms and also results in new online learning algorithms.